System and method for identifying and interpreting repetitive motions

ABSTRACT

A motion tracking system monitors the motions performed by a user based on motion data received from one or more sensors. The motion tracking system may include a motion tracking device with one or more sensors, a smart device with one or more sensors and/or a server. As the user interacts with the motion tracking system or smart device the motion data generated by one or more sensors is processed by a software application. The software application generates interpreted data based on the motion data and contextual data such as the equipment being used by the user. Feedback is then provided to the user during and/or after the user has performed a motion or a set of motions. The feedback provided to the user may be visual, audio or tactile. The application may be used to monitor a routine in a sporting, fitness, industrial or medical environment, for example.

RELATED APPLICATIONS

The present application is a continuation of U.S. application Ser. No.14/213,935, filed Mar. 14, 2014, which claims the benefit of U.S.Provisional Patent Application No. 61/792,601, filed on Mar. 15, 2013,U.S. Provisional Patent Application No. 61/873,339, filed on Sep. 3,2013, and U.S. Provisional Patent Application No. 61/873,347, filed onSep. 3, 2013, which are all incorporated by reference herein in theirentirety.

This application is related to U.S. Patent Application titled “Systemand Method for Monitoring Movements of a User”, filed on Mar. 14, 2014,the contents of which are hereby incorporated by reference.

FIELD OF THE EMBODIMENTS

The disclosure generally relates to the field of tracking usermovements, and in particular to monitoring and quantifying repetitiveand non-repetitive movements made by a user.

BACKGROUND

Motion processing and wireless communication technology allows people totrack things such as their sleeping patterns and the amount of stepsthey walk each day. However, motion capturing devices and functionalityhave not seen much success in the marketplace because of limits in thefunctions that can be performed and movement that can be monitored, forexample.

SUMMARY

Embodiments include a motion tracking system that monitors the motionsperformed by a user in real time, based on motion data received from oneor more sensors. The motion tracking system may include a motiontracking device with one or more sensors, a smart device with one ormore sensors and/or a server, for example. The user may wear the motiontracking device and or/carry the motion tracking device or the smartdevice while performing motions. As the user interacts with the motiontracking system or smart device the motion data generated by one or moresensors is processed by a software application. The software applicationmay be present on the smart device, the server, and/or the motiontracking device.

The software application generates interpreted data based on the motiondata and contextual data such as the equipment being used by the user.The interpreted data may include the performance of the user as the userperforms a motion and/or feedback provided to the user during or afterthe user performs a motion or set of motions. The software applicationidentifies the movement being performed by the user based on featurespresent in one or more signals of the motion data. The softwareapplication may count and or generate motion metrics associated with theperformance of the user as a user performs a motion. The interpreteddata is then provided to the user during and/or after the user hasperformed a motion or a set of motions. The feedback provided to theuser may be visual, audio or tactile, for example.

Based on the motion data, the software application may also determinethe form of the user as a user performs a motion, e.g., the form of theuser performing a particular exercise, motion, etc. The application maycompare the motion data generated by the sensors with a set of properform and improper form data and/or templates of the identified motion todetermine whether the user is performing a motion with improper form.

The application may also identify whether the user is demonstratingstrain while performing a motion, and may actively modify the routinesuggested to the user. The application may also modify the routinesuggested to the user based on comparing the motion data to historicaldata representing the performance of the user with respect to a motion.The application may use contextual data such as the location of the userand the user's calendar information to further enhance the fitnessexperience offered to the user.

As the software application is monitoring the user's movements,evaluating and keeping track of qualitative and quantitative metricssuch as the current exercise being performed by the user, the number ofrepetitions performed by the user and the form of the user, all in realtime and/or after the user has performed the motion, a set of motions,multiple sets of motions and/or one or more routines. Thus, the userdoes not have to provide input to the application by interacting withthe smart device or the motion tracking device. Hence, the user has thefreedom to perform the workout at his/her own pace, without theinterruption of periodically providing user input to the application viathe smart device or the motion tracking device.

The application may be used by coaches and physical therapists tomonitor the fitness of athletes and patients. The application may alsobe used in industrial or medical environments to actively monitor theroutine followed by a user.

The features and advantages described in the specification are not allinclusive and, in particular, many additional features and advantageswill be apparent to one of ordinary skill in the art in view of thedrawings, specification, and claims. Moreover, it should be noted thatthe language used in the specification has been principally selected forreadability and instructional purposes, and may not have been selectedto delineate or circumscribe the inventive subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings presented herein are for the purposes of illustration, theembodiments are not limited to the precise arrangements andinstrumentalities shown.

FIG. 1 is a perspective view of a motion tracking system, according toone embodiment.

FIG. 2 is a flowchart illustrating one implementation of the motiontracking system, according to one embodiment.

FIG. 3 is a flowchart illustrating the motion tracking system monitoringuser movements, according to one embodiment.

FIG. 4 illustrates repeated and non-repeated movements present in theprocessed signal, according to one embodiment.

FIG. 5 is a flowchart illustrating the motion tracking systemidentifying user movements based on motion data, according to oneembodiment.

FIG. 6 illustrates an implementation of the motion tracking system as afitness trainer, according to one embodiment.

FIG. 7 illustrates the motion tracking system determining the form of auser while the user is performing a motion, according to one embodiment.

FIG. 8 illustrates an implementation of displaying feedback to the user,according to one embodiment.

FIG. 9 illustrates a method by which the application generates good formand bad form movement templates and the feedback associated with eachtemplate, according to one embodiment

FIG. 10 represents a sample of motion data processed by the applicationas a user is performing a motion, according to one embodiment.

FIG. 11 illustrates the application dynamically adjusting the workoutsuggested to the user, according to one embodiment.

The figures depict various embodiments for purposes of illustrationonly. One skilled in the art will readily recognize from the followingdiscussion that alternative embodiments of the structures and methodsillustrated herein may be employed without departing from the principlesof the embodiments described herein.

DETAILED DESCRIPTION

Embodiments are now described with reference to the figures where likereference numbers indicate identical or functionally similar elements.Also in the figures, the left most digit(s) of each reference numbercorresponds to the figure in which the reference number is first used.

FIG. 1 is a perspective view of a motion tracking system 100, accordingto one embodiment. In one aspect of an embodiment, as discussed indetail with reference to the figures below, a user 23 wears a motiontracking device 24 while such user 23 is performing motions such asweight training, walking and cardiovascular movements and/or liftingobjects. The motion tracking system 100 monitors the motion of a user inreal time. In one embodiment, the motion tracking device 24 includes amotion processing unit 5 which measures a repetitive movement 32 or anon-repetitive movement 33 performed by the user 23. The motionprocessing unit 5 includes one or more sensors, such as an accelerometer6, a gyroscope 7 and/or a magnetometer 8. The motion data 25 measured bythe sensors and the motion processing unit 5 may be used to monitor themovements of a user in real time.

The motion data 25 is transmitted to an auxiliary smart device 18running a software application 19. The application 19 analyzes themotion data 25 and generates an interpreted data 28 to provide to theuser 23. The application 19 also provides the user 23 with feedbackregarding the user's movements. For example, the application 19 mayanalyze motion data 25 related to a user performing an exercise andprovide feedback to the user 23 in real time. The feedback may includethe quality of the form of the user's motion, recommendations for otherexercises or the performance of the user. Motion data 25 is also, in oneaspect, analyzed by the application 19 along with contextual data 26.The contextual data 26 may be gathered from a number of sources such asother application data on the smart device 19 (e.g., geographicallocation, time of day, etc) or from capturing devices such as a cameraor a RFID tag/reader 2. Associating contextual data 26 with motion data25 allows the application 19 on the auxiliary smart device 18 to provideadditional information to the user related to the health, fitness ormotions being performed by the user.

In one embodiment the motion tracking device 24 houses a microcontroller1. Microcontroller 1 may be a small computer on a single integratedcircuit containing a processor core, memory, and programmableinput/output peripherals which manage multiple inputs and outputs thattake place within the motion tracking device 24. Microcontroller 1 mayreceive direct inputs from user input 11 to power the motion trackingdevice 24 on/off, to trigger data visualization sent to a display 10 andto turn down the volume on a speaker 13. In one embodiment,microcontroller 1 is coupled to other components via a single printedcircuit board or flexible circuit board.

In one embodiment the motion processing unit 5 is connected to themicrocontroller 1 and a regulated power supply 17. Motion processingunit 5 includes multiple sensors which measure user 23's repetitivemovements 32 and non-repetitive movements 33. Each component within themotion processing unit 5 measures a type of motion. For example, theaccelerometer 6 detects changes in orientation and acceleration of themotion tracking device 24, the gyroscope 7 measures the angular velocityand the magnetometer 8 measures the strength and direction of magneticfields. Hence, the sensors in the motion processing unit 5 allow themotion tracking device 24 to track the movements performed by the user23. When motion data 25 is recorded by the motion processing unit 5, itmay be sent to one or more locations. In one aspect of the presentdisclosure, motion data 25 is sent from the motion processing unit 5 tothe microcontroller 1, where motion data 25 may be temporarily stored inan onboard memory 9. In one embodiment, motion data 25, along with thepossible contextual data 26, are sent to smart device 18 via acommunications module 4.

In one aspect of the present disclosure, motion data 25 may be sentdirectly to smart device 18 by the communications module 4.Communications module 4 is, in one embodiment, a Bluetooth module, butcould also include Wi-Fi, zigbee, or any other form of wirelesscommunication, either in conjunction with or instead of Bluetooth. Thecommunications module 4 is coupled to other components such as themicrocontroller 1 and a regulated power supply 17. The regulated powersupply 17 regulates the power transferred to different components from abattery 16.

In one embodiment, a recharge management 15 component acquires powerfrom a USB input 12 and delivers it to the battery 16. In anotherembodiment, the recharge management 15 component acquires power fromother forms of input and is not limited to acquiring power from the USBinput 12. Battery 16 may be, but is not limited to, a rechargeable ornon-rechargeable lithium ion battery, a rechargeable or non-rechargeablenickel metal hydride battery, a rechargeable or a non-rechargeablealkaline battery. In one embodiment, the battery 16 sends the powerneeded to the regulated power supply 17. The regulated power supply thendistributes power to all components which need it. These componentsinclude but are not limited to the microcontroller 1, communicationsmodule 4, motion processing unit 5, memory 9, display 10, speaker 13 anda vibrator 14. In one aspect of the present disclosure, the motiontracking device 24 may be powered using solar cells mounted on a surfaceof the motion tracking device 24.

In one embodiment the speaker 13 is connected to the microcontroller 1and/or the regulated power supply 17. The speaker 12 receives audio cuesfrom microcontroller 1. Sound from speaker 13 is emitted through one ormore speaker ports. Speaker ports 34 may be, but not limited to,perforations located on the surface of the motion tracking device 24.Microcontroller 1 may also use the vibrator 14 to send tactile cues tothe user 23. Vibrator 14 can be an off-axis motor which when triggeredby microcontroller 1 creates a vibrating sensation for user 23. Vibrator14 is connected to microcontroller 1 and regulated power supply 17,power is pulled from battery 16 to power the component.

In one embodiment, the motion tracking device 24 is a wearable apparatusintended to be worn by the user 23 while performing repetitive movements32. Motion tracking device 24 may be wrapped around a limb or part ofthe user 23's body using a strap band and a strap connector (not shownin FIG. 1). Motion tracking device 24 has a surface which may beintended to communicate and/or display data to the user 23 viacomponents such as display 10 and speaker ports 34. Display 10 is avisual screen that the user 23 can read. Functions pertaining to display10 may be, but are not limited to, displaying interpreted data 28,managing interpreted data 28, displaying battery life and managing thesettings installed on motion tracking device 24 such as the volumeassociated with speaker 13. The display 10 may be, but is not limitedto, an LED display, an LCD display, an electronic ink display, plasmadisplay or ELD display and may be, but not limited to, being mounted onthe surface of the motion tracking device 24. The speaker port is acollection of perforations that emit audio cues given off by speaker 13.The speaker port may be, but is not limited to, being located on thesurface of the motion tracking device 24, it may be located in otherlocations such as on a side wall of the motion tracking device 24. Userinputs 11, for example, buttons, protrude through the surface 36 ofmotion tracking device 24. User inputs 11 may be located on any otherexterior surface of motion tracking device 24 such as side wall.Functions of user inputs 11 may be, but are not limited to, scrollingthrough interpreted data 28 on display 10, turning motion trackingdevice 24 on/off, managing interpreted data 28 via display 10,visualizing battery life, displaying notifications regarding motiontracking device 24 and managing volume levels of speaker 13. Motiontracking device 24 is charged via charging port. The charging port maybe, but is not limited to being, located on the side wall of the motiontracking device 24. The charging port may be a micro USB input 12, amini USB port, an audio input, or any other means of transferring power.

The motion tracking device 24 may be, but not limited to, beingmanufactured out of a flexible composite, so it may naturally convertfrom laid out flat, to wrapped around a limb. In one aspect, motiontracking device 24, including the strap bands and the surface of themotion tracking device 24, is injection-molded out of a water resistantsilicone, capable of various ranges of motion without causing stress onthe silicone or the internal components. According to one aspect of thepresent disclosure, the strap bands may be made of rugged textile,capable of various ranges of movement. The strap connectors 41 havecontact surfaces which may be, but not limited to a Velcro™ adhesive,magnetic tape a snapping mechanism or any other components thereof. Inone aspect of the present disclosure, the strap bands are embedded withmagnets which then create the resulting connection between each strapband 40.

The motion tracking device 24 may be of various lengths and sizes,dependent on the part of the body from which motion data 25 is beingrecorded. In one aspect of the present disclosure, strap bands 40 may becapable of stretching to meet the length requirements necessary tosecure motion tracking device around user 23 via strap connector 41.

In one embodiment, the motion tracking device 24 houses components forcapturing contextual data 26 such as a camera or a RFID tag reader 2.The camera captures images or videos of the environment the user is inor items the user is interacting with. For example if a user isperforming a curl, the camera may capture an image of the dumbbell beingused by the user to perform a curl, as the user is performing a curl.The microcontroller 1 receives the image from the camera and sends theimage to the software application 19. The software application 19 mayprocess the captured image (contextual data 26) and generate interpreteddata 28 identifying the weight of the dumbbell being used by the user.In another example, an RFID tag reader 2 may capture an RFID tagassociated with the dumbbell being used by the user to perform a curl.The microcontroller 1 receives the RFID tag identifier from the RFID tagreader 2 and sends the RFID tag to the software application 19. Thesoftware application 19 may process the RFID tag (contextual data 26)and generate interpreted data 28 identifying the weight of the dumbbellbeing used by the user, as identified by the RFID tag. In an alternateembodiment, the user may input the contextual information via thesoftware application 19 and/or the motion tracking device 24.

In one embodiment, motion data 25 and contextual data 26 are sent to thesoftware application 19 installed onto smart device 18. Softwareapplication 19 interprets motion data 25 and contextual data 26 intointerpreted data 28. In one embodiment, the interpreted data 28 mayinclude the user's 23 movements, pauses in movement, collections ofmovements and any other contextual information related to the user'smovements. Interpreted data 28 can also be interpretations of contextualdata 26, which can also, in one aspect, include estimates of thecalories burned by the user during a given exercise reflected by a givenset of motion data 25 using a piece of equipment identified by a givenset of contextual data 26. In another embodiment, the interpreted data28 includes the performance of the user during a set of motions andfeedback provided to the user during and/or after the user performs aset of motions.

The smart device 18 may be any device capable of accepting wireless datatransfer such as a smartphone, tablet or a laptop. In one embodiment thesmart device 18 has computing power sufficient to run the softwareapplication 19. Persons having skill in the art will realize thatcommunication is not necessarily direct between the motion trackingdevice 24 and the smart device 18, and could instead be indirect, viaone or more intermediary devices and/or via a network such as theInternet. The software application 19 interacts with the smart device 18through a smart device API. The software application 19 receives motiondata 25 from the motion tracking device 24 by using the smart device APIto interact with the communication module 4. The software application 19may be adapted to interact with a variety of smart device APIs. Thiswould allow the software application 19 to function on a variety ofsmart device platforms 18 each having their own smart device API. Hence,the user is not restricted to a specific smart device 18 in order to beable to use the application 19.

In one embodiment the software application 19 is hosted or installed onthe motion tracking device 24. In this embodiment, the softwareapplication 19 may be executed by the processor on the microcontroller1. Hence, the analysis of the motion data 25 may be performed by thesoftware application 19 on the motion tracking device 24, independent ofthe smart device 18 or in combination with the smart device 18 and/or aremote processing device, e.g., server 21. In another embodiment, thesoftware application may be installed on a device with at least onesensing component, such as a smartphone. The software application 19 inthis embodiment, may use motion data 25 provided by the sensors on thedevice to generate interpreted data 28, and not on the pairing of thesmart device 18 and the motion tracking device 24. For example, theapplication 19 installed on a smartphone 18, may use the motion data,generated by the accelerometer 6 on the smart phone to determine thenumber of steps taken by the user as the user 23 walked from his/herhouse to work. Hence the motion tracking system 100 is not restricted tothe coupling of a smart device 18 and a motion tracking device 24, andcan be performed in any number of steps with any number of devicesinvolving the transfer of motion data 25 to the software application 19.In alternate embodiments, sensor information from multiple devices,e.g., smart device 18 and motion tracking device 24, can be used bysoftware application 19.

In one embodiment the interpreted data 28 is sent from the smart device18 to a remote processing device (cloud based device and system), e.g.,server 21 via a wireless data transfer or a network. For ease ofreference, server 21 will be used in this description, but any remote,e.g., cloud based, processing device including multiple devices such asa remote database, storage, memory, processor(s) can be used. The server21 can be any remote processing device. The Server 21attaches/correlates/identifies the interpreted data 28 to a user profile29. The user 23 may then review, access and/or visualize the interpreteddata 28 history associated with their user profile 29 via any devicecapable of wireless data transfer such as, without limitation, a smartphone, a tablet, a motion tracking device, or a computer, using adedicated software application or a web browser to display theinterpreted data 28. In another embodiment, the interpreted data 28 isalso relayed back to the user 23 through software application 19installed on the smart device 18 or on the motion tracking device 24.Interpreted data 28 may be displayed by the software application 19 forthe user 23 to see during and/or following the user 23 performingmovements. Feedback regarding the interpreted data 28 may be provided tothe user 23 in real time in a number of ways. In one example visualfeedback is provided to the user 23 either on the smart device 18 or onthe display 10 of the motion tracking device 24. In another example,audio feedback is provided to the user through speakers on the smartdevice 18 or on the motion tracking device 24. Tactile feedback may alsobe provided to the user through the vibrator 14.

In one embodiment, the software application 19 may be stored on theserver 21. The software application 19 on the server 21 may analyze themotion data 25 sent to the server and generate the interpreted data 28to associate with a user profile 29. For example, in the instance thatthe user 23 would like to save the power consumed by the smart device18, the smart device 18 may send motion data 25 received from the motiontracking device 24 to the server 21 for processing. Hence, theprocessing of the motion data 25 by the software application is notlimited to taking place on the smart device 18 or on the motion trackingdevice 24.

In one embodiment, the software application 19 or other code stored onthe smart device 18 or the motion tracking device 24 may regulate thepower consumed by the sensors by turning on or off one or more sensors.In one example, the sensors are turned off when the user has notactivated or moved the device 24. In another example, one or moresensors are turned on or off for particular movements performed by theuser.

FIG. 2 is a flowchart illustrating one implementation of the motiontracking system 100, according to one embodiment. In this embodiment theuser is using the motion tracking system 100 as an artificial aide and amonitoring unit while performing a fitness routine. The user activates205 the motion tracking device 24 or the application 19 on the motiontracking device 24 by either pressing user input 11 or moving the motiontracking device 24. In one example the application 19 on the motiontracking device 24 identifies that the user has activated the devicebased on motion data 25 received from the sensors.

The user then begins the fitness routine by either following a routinesuggested by the application 19 or by following a routine the user wouldlike to perform. For example, the routine suggested by the application19 may include 3 sets of hammer curls using 30 pound dumbbells with arest period of 60 seconds between each set, followed by 4 sets of 20crunches with a rest period of 30 seconds between each set. As the userperforms the routine, the application 19 monitors a number ofcharacteristics related to the movements performed by the user based onthe motion data 25. For example the application 19 determines andmonitors 215 the type of exercise being performed by the user, thequality of the form of the user as the user is performing the exerciseand/or the number of counts or repetitions performed by the user. In oneembodiment, the application 19 suggests and monitors 215 the rest timeobserved by the user in-between sets of exercises as the user goesthrough the fitness routine.

The application 19 may also provide feedback 220 to the user inreal-time as the user performs the fitness routine. For example thevibrator 14 on the motion tracking device 24 may vibrate, notifying theuser of bad form as the user is performing a curl. In another examplethe feedback includes, charts and tables displayed on the display 10 ofthe motion tracking device 24 describing the performance of the userthrough the fitness routine.

In one embodiment the application 19 sends 225 the interpreted data anda performance data to the server 21. The performance data may includestatistics describing the performance of the user throughout the fitnessroutine, or quantitative metrics (e.g., percentage of routine completed,goals reached, repetitions of each exercise, etc) evaluating the fitnessroutine performed by the user. The server 21 then associates or attaches230 the performance data and/or the interpreted data 28 to the user'suser profile 29.

FIG. 3 is a flowchart illustrating the motion tracking system 100monitoring user movements, according to one embodiment. The application19 monitors the movements made by the user based on the raw real timemotion data 25 obtained 305 from the sensors. The sensors generate rawreal time motion data 25 based on the movements of the user. Forexample, the accelerometer 6 generates acceleration data and change inacceleration data based on the relative movement of the device 18, 24 onthe user.

The application 19, then processes 310 the real time motion dataobtained 305 from one or more of the sensors or the motion trackingdevice 24. Processing 310 the raw real time data or signal removes thenoise and other irrelevant features carried by the signal. In oneembodiment a low pass filter is used to filter out the noise in the rawsignal obtained 305 from the sensors. In another embodiment a movingaverage filter is used to filter out the noise in the raw signalobtained 305 from the sensors. It is understood that other filters canbe used to increase the signal-to-noise ratio of the raw signals.

In one embodiment the application 19 determines 315 a classification ofthe movement performed by the user based on one or more processed realtime signals. Classifying the movement performed by the user isimportant as it helps the system identify and understand the movementbeing performed by the user. For example, the application 19 firstdetermines 315 that the user is performing a curl, prior to identifyingthe characteristics associated with the user performing the curl, suchas the form of the user's movements with respect to that of a correctcurl movement.

In one embodiment, a classification algorithm may be a machine learningalgorithm, a pattern recognition algorithm, a template matchingalgorithm, a statistical inference algorithm, and/or an artificialintelligence algorithm that operates based on a learning model. Examplesof such algorithms are k-Nearest Neighbor (kNN), Support Vector Machines(SVM), Artificial Neural Networks (ANN), and Decision Trees.

In one embodiment, after the application classifies 315 the movementbeing performed by the user, the application 19 quantifies 320characteristics of the movement being performed by a user such as thecount of the number of repetitive movements made by the user todetermine the repetitions of a movement performed by a user. Forexample, the application 19 determines the number of times a user hasperformed a curl during a given set of curls, based on the number ofrepetitive movements (that have been classified as a curl) performed bythe user. In one embodiment, the application 19 determines the number ofreal peaks present in a rolling window of one or more signals. A realpeak may be determined based on the amplitude of the peak relative tothe whole signal and/or other contextual information such as theexpected pattern of peaks or duration of peaks for the classified oridentified movement being performed by the user. For example, theapplication 19 may have identified that the user is performing a curl.Based on this information, real peaks may be known to appear in therolling window of the z-axis of the accelerometer 6 signal above anamplitude of 0.6 G and over a time period n as a user is performing acurl. Similarly real peaks may be known to appear in the rolling windowof the y-axis of the gyroscope 7 signal above an amplitude of 0.5radians/sec and over a period of 2n as a user is performing a curl.Hence, the application 19 may count the number of real peaks present inthe z-axis accelerometer 6 signal as 1 per time period of n, and thosepresent in the y-axis gyroscope 7 signal as 2 per period of 2n, therebycounting the number of curls performed by the user.

In another embodiment, the application 19 may quantify 320 othercharacteristics of the movement being performed by the user such as thespeed of the movement being performed by the user. The application 19may determine the time period over which a peak or valley ormorphological feature in one or more signals occurs to determine therate at which each repetitive movement is performed by the user. Longertime periods may correspond to slower movement speeds, and shorter timeperiods may correspond to fast movement speeds. The application 19 maythus quantify 320 a number of characteristics associated with themovements performed by a user based on the morphological featurespresent in one or more signals.

FIG. 4 illustrates repeated 405 and non-repeated 410 movements presentin the processed signal, according to one embodiment. Referring to FIG.4 with respect to the method illustrated in FIG. 3, the application 19counts the repetitive movements 405 performed by the user during afitness routine. For example, if the repeated movements 405 were that ofcurls performed by the user, the application 19 would determine that theuser performed 5 repeated movements or 5 curls. The application 19,differentiates between the non-repeated movements 410 represented by aportion of the processed signal and the repeated movements 405represented by a different portion of the processed signal.

In one embodiment, the application 19 identifies groups of repeatedmovements 405 performed by a user. For example, the fitness routinesuggested by the application may include the user receiving instructionsto perform 3 sets of 5 curls with a rest time of 30 seconds between eachset. The application 19, based on the processed real time signal firstidentifies and classifies the user's movements as curls. Then theapplication 19 is notified of the user performing the first set ofcurls, based on the user performing repetitive curl movements 405. Afterthe application 19 has recorded group 1 (415) comprising of 5 curls, theapplication 19 also monitors the transition time 1 (430) or the resttime, represented by the non-repeated movements 410 between groups 1(415) and 2 (420). The application 19 then monitors group 2 (420)comprising of 5 curls, and the transition time 2 (435) between group 2(420) and group 3 (425). The application 19 identifies that the user hasfinished the 3 sets of curls once the application has finishedmonitoring group 3 (425), the last set of curls performed by the user.Hence, the application 19 monitors the fitness routine followed by theuser based on the processed real time signal, representing the movementsperformed by a user.

FIG. 5 is a flowchart illustrating the motion tracking system 100identifying user movements based on motion data 25, according to oneembodiment. The application 19 determines the movement performed by theuser based on one or more processed real time signals. The application19 extracts 505 a set of statistical or morphological features presentin a rolling window of one or more processed signals. Features mayinclude amplitude, mean value, variance, standard deviation of thesignal, the number of valleys and/or peaks, the order of valleys and/orpeaks, the amplitude of valleys and/or peaks, the frequency of valleysand/or peaks and/or the time period of valleys and/or peaks in one ormore processed signals. For example, while performing a curl, the z-axisof the accelerometer 6 might record a repetitive pattern of a singlepeak over a time period n, followed by a single valley also over a timeperiod n. The y-axis of the accelerometer 6 might record a repetitivepattern of a valley between 2 peaks during the same time period n. Theextracted features are used by the classification algorithm to detectthe movement being performed by the user.

In one embodiment the application 19 applies a template matchingalgorithm to identify repetitive features (such as peaks and valleys inthe signal) in a rolling window of one or more processed signals. Theapplication 19 compares the repetitive features in the rolling window ofone or more processed signals to a set of movement templates 27 storedin a movement template database 22 on the smart device 18 or the motiontracking device 24. Based on the comparison, the application 19 thenselects the movement templates 27 in the database 22 having repetitivefeatures most similar to or most closely matching those present in oneor more processed signals. Based on one or more or a combination of theselected templates the application 19 identifies and classifies 515 themovement being performed by the user. For example, the application 19compares the repetitive features present in the z-axis of theaccelerometer 6 signal and the y-axis of the gyroscope 7 as a user isperforming a curl, with the movement templates 27 stored in the movementtemplate database 22. The application 19 selects a z-axis accelerationsignal movement template 27 and a y-axis gyroscope 7 signal movementtemplate 27 similar to that of the recorded signals. The application 19then identifies 515 that the user is performing a curl, as the twomovement templates 27 selected by the application 19 are known to beassociated with a curl movement. One example of template matchingalgorithms is cross correlation algorithm. Another example is dynamictime warping.

In one embodiment, the application 19 guides the user through a fitnessroutine. As the application 19, is guiding the user through the fitnessroutine, the application 19 is aware of the repetitive movement beingperformed by the user. Thus, the application 19, may verify the movementidentified by the application 19 based on the recorded motion data 25with the movement the application 19 expects the user to perform basedon the fitness routine. In a second embodiment, as the application 19 isaware of the movement being performed by the user, the application 19 nolonger needs to identify the movement being performed by the user basedon the motion data, and hence begins to count the repetitive featurespresent in one or more processed signals to determine the repetitionsperformed by the user. In a third embodiment, as the application 19 isaware of the movement being performed by the user, the application 19may compare the recorded motion data to a subset of movement templates27 in the movement template database 22, wherein the subset of movementtemplates 27 represent templates related to the movements theapplication 19 expects the user to perform. For example, if theapplication 19 is aware that the user is currently performing a curl aspart of a fitness routine, the application 19 would compare the recordedmotion data 19 with that of movement templates 27 associated with thecurl classification of movements.

In one embodiment, the application 19 determines 510 the statisticalcharacteristics such as a mean or standard deviation associated with therepetitive features in one or more signals. For example, the application19 may determine 510 the mean of the amplitude of the peaks recorded inone or more signals, while the user is performing curls. If the mean isfound to be relatively greater than that of the expected threshold forcharacterizing real peaks, the application 19 may raise the weight forthe next set of curls suggested to the user as a relatively higher meanimplies that the user was able to perform a current curl easier (at afaster rate) than that is expected. In another embodiment, theapplication may determine the standard deviation of the amplitude andfrequency of the peaks recorded in one or more signals, while the useris performing curls. If the standard deviation is found to be outside ofan expected range of standard deviation values for a curl action, it ispossible that even though the pattern of features may have beenidentified to match a curl, the user may not really be performing acurl, but may be performing a different motion similar to a curl. Hence,the statistical characteristics of the features in one or more signalsprovide additional information towards the identifying 515 the movementperformed by the user.

In another embodiment, the application 19 uses machine learningalgorithms to detect movements, and/or classify or identify 505movements performed by a user in a rolling window of one or more signalsbased on the repetitive features or morphological features present inone or more signals. An example of a recognition and a learningalgorithm that may be used is described in Ling Bao et. al, “ActivityRecognition from user-Annotated Acceleration Data”, which isincorporated by reference herein in its entirety.

FIG. 6 illustrates an implementation of the motion tracking system 100as a fitness trainer, according to one embodiment. The application 19guides the user through one or more workouts over a period of time. Aworkout is comprised of a combination of one or more exercises aimed atachieving a fitness goal. The fitness goal may include increasing thestrength of the user, reducing the weight of the user, increasing thestamina of the user, or any other form of beneficial physicalimprovement or maintenance of the user's physical condition.

In one embodiment, the user may create workouts by selecting one or moreexercises from a set of exercises available in an exercise database 22stored on the smart device 18, the server 21 or the motion trackingdevice 24. The application 19 associates the workouts created by orselected by a user with the user's profile 29. The user may also modifyother attributes related to a workout such as the frequency of days in aweek a certain workout is performed or the time period, sets andrepetitions of exercises being performed during a workout. For example auser may choose to create a work out involving 5 exercises related tostrengthening legs on Monday, 5 exercises related to strengthening theupper body on Wednesday, and an hour of running on Friday.

In one embodiment, the workouts may be planned and suggested bydifferent professional trainers 615, 620, 625. For example trainer 1(615) may specialize in designing workouts for strength training,trainer 2 (620) may specialize in designing workouts for weight loss,and trainer 3 (625) may specialize in designing workouts for cardiotraining. The user may choose to follow workouts suggested by a singletrainer or by a combination of the trainers. The application 19associates the selected trainers and workouts with the user's profile29.

In one embodiment, the user selects a workout from one or many workoutssuggested by trainer 3 (625) via the application 19. The user thenfollows feedback 630 or directions suggested by the workout eithervisually through the display 10 of the motion tracking device 24 or thedisplay of the smart device 18, or by listening to audio instructionsfrom the motion tracking device 24 or the smart device 18. Feedback canalso be in the form of a video explaining the proper form of a motion,e.g., exercise, that can be displayed on the motion tracking device 24,smart device 18 or other display either at the time or at a later time,e.g., a message (email, text, social media post, can include a link toor an embedded video with feedback. As the application 19 is monitoringthe user's movements, keeping track of qualitative and quantitativemetrics such as the current exercise being performed by the user, thenumber of repetitions performed by the user and the form of the user,all in real time, the user does not have to provide input to theapplication by interacting with the smart device or the motion trackingdevice. Hence, the user has the freedom to perform the workout athis/her own pace, without the interruption of periodically providinguser input to the application via the smart device or the motiontracking device.

In one embodiment, the trainers 615, 620, 625 may monitor the workoutperformed by the user from a remote location, as the motion data 25,interpreted data 28 and performance data 25 are attached to the user'sprofile on the server 21. Hence, the application 19 provides for a userto maintain a relationship with a personal trainer irrespective of thelocation of the user and the personal trainer. Furthermore, the trainernow also has access to multiple other forms of motion data 25 describingthe fitness of the user. For example, the trainer can determine thenumber of steps a user walks or performs during a given time of day,such as on the way to work, or the time spent by a user sitting down atwork or performing repetitive motions at work. This information can helpthe trainer create controlled workouts that are tailored to each userbased on the various activities performed by a user over a period oftime.

In one embodiment, the trainer can add additional workouts to thedatabase 22 via the application 19. The additional workouts may bestored on a remote server 21 or on the trainers' smart device 18. Theuser may then access the workouts from the server 21, either by pullinga selected set of workouts from the server 21 or by allowing theapplication 19 to automatically update the workouts followed by the userbased on a set of work out preferences selected by the user.

FIG. 7 illustrates the motion tracking system 100 determining the formof a user while the user is performing a motion, according to oneembodiment. The application 19 monitors the form of a user as the useris performing a motion and provides feedback 630 to the user in realtime and/or after the user has performed the motion. The application 19compares 715 one or more of the recorded movement signals 710 withcorrect movement template signals 705, based on the identified motionbeing performed by the user, and then provides the user with feedback630. As shown in FIG. 7, the difference in the user's movement and thecorrect movement can be seen as the variations 720 at the peak of thesignals. In one embodiment, if the application determines that there issignificant difference between the correct movement template signal 705and the recorded movement signal 710 the application notifies 630 theuser of improper or bad form. Improper or bad form represents the userperforming a motion incorrectly or poorly such that the user may not bereaping the full extent of benefits by performing the motion, or theuser may potentially be harmed if the user continues to perform themotion incorrectly. The application 19 may determine improper or badform by comparing the recorded movement signal 710 with improper formtemplates (not shown) for the identified movement. The improper formtemplate that most closely matches the recorded movement signal 710enables the application to identify the improper form based upon thatimproper form information associated with the improper form templates.In an alternate embodiment, as described below, the improper formtemplate can be part of the initial template comparison. For example,the application 19 may incorporate the improper form templates with theinitial template comparison, as the user begins to demonstrate strainwhile performing a motion, or when the user approaches the end of a setof an exercise.

In one embodiment visual feedback is provided to the user 23 either onthe smart device 18 or on the display 10 of the motion tracking device24. Alternatively, visual feedback may be stored remotely for access ata later time by any computing device. For example, the feedback may bestored or referenced, e.g., an html link on a social networking page, avideo sharing website, e.g., YouTube™, and/or other manner of providingvideo, for example. The application 19 notifies the user of improperform, and displays to the user a series of images or videos representingthe correct form of performing the movement. In another example, audiofeedback is provided to the user through speakers on the smart device 18or on the motion tracking device 24. Tactile feedback may also beprovided to the user through the vibrator 14.

FIG. 8 illustrates an implementation of displaying feedback to the user,according to one embodiment. In one embodiment, the application 19compares one or more recorded movement signals to a set of good form andimproper form movement templates 27 associated with an identifiedmotion. Once the application 19 determines the type of improper formbeing performed by the user, the application 19 provides feedback 810notifying the user of the type of improper form and how to avoid thetype of improper form being performed by the user. For example, theapplication 19 compares the recorded movement signal with a set ofmovement templates 27 known to represent improper forms of performing acrunch. The application 19 identifies the improper form performed by theuser and provides feedback 810 notifying the user that the user isbouncing too much between crunches. In one embodiment, the application19 also displays to the user via a display on the smart device 18 or onthe motion tracking device 24, a video of proper form while performing acrunch. In another embodiment, the application monitors the motion of auser and notifies the user when the user is performing the motion withgood form. In one embodiment the application 19 may also provide theuser with performance feedback 805 illustrating the performance of theuser during and/or after the user has performed a set of motions or afitness routine.

FIG. 9 illustrates a method by which the application 19 generates goodform and improper form movement templates 27 and the feedback 630associated with each template, according to one embodiment. Good formmovement templates 27 may be generated by those who are known to beskilled in performing a motion with correct form. For example, certifiedpersonal trainers may be monitored for a period of time as they performa variety of exercises and fitness routines. The signals from thevarious sensors generated by the certified personal trainers are storedin the movement template database 22 as movement templates 27representing good form. The application 19 compares the movement signalsgenerated by a user against the good form movement templates 27 todetermine whether the user is performing an exercise with good form.

In one embodiment, the application 19 may receive motion data 25representing improper form from a variety of users performing the samemotion. For example, the application 19 may receive motion data 1 (940)from user 1 (905), motion data 2 (945) from user 2 (910), motion data 3(950) from user 3 (915), and motion data 4 (955) from user 4 (920) allof which represent different types of improper form associated with acurl motion. The application 19 may then send different feedback 630 toeach of the users with information describing how they may avoid theimproper form. The feedback 630 would suggest to each user a differentway of performing the motion so as to avoid improper form. Theapplication 19 may then monitor the change in motion data generated byeach of the users to determine whether the users now perform the motionwith good form. User 1 (905) and user 3 (915) may now perform a curl ina motion that represents good form. The application 19 would thenassociate the feedback 630 provided to user 1 (905) with the originalmotion data 1 (940) representing improper form. The application 19 wouldthen generate and store a movement template based on motion data 1 (940)representing improper form. Feedback 630 associated with motion data 1(940) can then be used by the application 19 to notify future usersperforming a curl motion similar to motion data 1 (940) of how tocorrect the improper form. Hence, the application 19 can generatemovement templates 27 representing improper form and determineappropriate feedback 630 to associate with each movement template so asto prevent a user from performing a motion with improper form. Asdescribed above, these improper form templates may be used initiallyalong with the good form templates as part of the process of identifyingthe motion and/or determining whether proper form was used.

In one embodiment, the application 19 detects whether a user is notfully performing the exercise motion, e.g., cheating, while performingan exercise motion. For example a user may have reached the portion ofthe workout that involves the user performing curls during a workout.However, as opposed to performing a set of curls, the user decided tomove the motion sensing device 24 back and forth causing the application19 to count down through the set of curls. In one embodiment theapplication 19 compares the motion data 25 generated by the sensors to aset of movement templates 27 representing cheat movements. If theapplication 19 identifies that the motion data 25 generated is similarto the set of cheat movement templates 27 the application 19 may notifythe user that the user is cheating during the fitness routine. Theapplication 19 may also notify the trainer that the user may havecheated through a set of curls assigned to a fitness routine or workout.

In another embodiment, the application 19 may also use statisticalanalysis techniques along with template matching to determine whether auser is cheating during a workout. For example, the application 19 maydetermine the mean or standard deviation of the amplitudes orfrequencies of peaks present in one or more signals of the processeddata. If, for example, the mean of the amplitudes was relatively higherthan historical data associated with the user performing curls and thecurls were performed in a period of time shorter than that normally(based on historical data) taken by a user, the application 19 mightidentify that the user cheated while performing that set of curls. Hencea combination of statistical techniques and template matching may beused to determine whether or not a user is cheating while performing anexercise.

FIG. 10 represents a sample of motion data processed by the application19 as a user is performing a motion, according to one embodiment. Theapplication 19 may determine whether or not a user is struggling toperform a motion or straining to perform a motion. For example, theapplication 19 may determine features present in one or more signalsassociated with a user demonstrating strain while performing an exerciseor a motion. As one example, the application 19 may determine based onprocessed signals 1 (1010), 2 (1020) and 3 (1030), whether or not a useris demonstrating strain while performing a set of curls. Processedsignal 1 (1010) may represent the acceleration measured by theaccelerometer 6 along the x-axis of motion, in a direction pointing fromthe elbow of the user down towards the dumbbell in the user's hand asthe user performs a curl. As the user raises the dumbbell, processedsignal 1 (1010) decreases in value until it reaches a valleyrepresenting the beginning of the decent of the user's arm. Processedsignal 2 (1020) may represent the acceleration measured by theaccelerometer 6 in a direction normal to the face of the dumbbell. Asthe user raises and lowers the dumbbell the value of processed signal 2(1020) remains within a controlled range. Processed signal 3 (1030) mayrepresent the acceleration measured by the accelerometer 6 in adirection normal to the palm of the user, as the user holds thedumbbell. As the user raises the dumbbell halfway, processed signal 3(1030), decreases in value until it reaches a valley, after which theprocessed signal 3 (1030) increases in value until the user hascompletely raised the dumbbell to the top of the curl action. As theuser lowers the dumbbell halfway, processed signal 3 (1030), decreasesin value until it reaches a valley, after which the processed signal 3(1030) increases in value until the user has completely lowered thedumbbell to the bottom of the curl action.

In one embodiment, the application 19 identifies that the user isperforming a curl based on repetitive features present in processedsignal 1 (1010) and processed signal 3 (1030). Using template matching,the application 19 may determine that the repetitive pattern of a peakin-between two valleys in processed signal 3 (1030) that occurs in thesame time period as a single valley in processed signal 1 (1010) matchesa movement template known by the application 19 to represent a curlmotion.

In one embodiment, as the user progresses through the set of curls, thetime period 1050 of the peak representing movement count 5 (1040) issignificantly greater than that representing movement count 4 (1050).The range of the amplitude of the vibrations or motion measured byprocessed signal 2 as the user performs the movement count 5 (1040) mayalso appear to be substantially greater than the relatively consistentrange of values measured earlier during movement counts 1, 2 or 3 (1040)by the accelerometer 6. The application 19 may compare the presence ofthe substantially greater time period 1050 in signal 1 (1010), and thesubstantially greater range of amplitude values in processed signal 2(1020) during count 5, with a known movement template representing theuser demonstrating strain, to identify that the user is demonstratingstrain while performing count 5 (1040) of the set of curls. Hence, theapplication 19 may use a combination of signals (processed signal 1(1010) and processed signal 3 (1030) to identify the motion or exercisebeing performed by the user, and a similar or different combination ofsignals (processed signal 1 (1010) and processed signal 2 (1020) todetermine whether a user is demonstrating strain while performing anidentified motion.

In one embodiment, the application 19 modifies a sampling rate at whichdata is sampled from the sensors in the motion tracking device 24 or thesmart device 18. In one example the sampling rate is modified based onthe exercise being performed by a user. In the instance a user isperforming an exercise with relatively fast repetitions such as bikingon a bicycle the sampling rate will need to be modified to one at afaster rate as compared to when a user performs an exercise withrelatively slow repetitions such as squats. The active control of thesampling rate is helpful in minimizing the power consumed by the motiontracking device 24 or the smart device 18 while performing functionsassociated with application 19.

In another embodiment, the sampling rate is increased as the user getsclose to the maximum number of repetitions in a set to more accuratelymeasure strain and/or improper form. The sampling rate can also increaseif the application 19 identifies that the user is starting to struggle,e.g., strain is detected, time between repetitions increases, etc. Forexample, and with reference to FIG. 10, the application 19 may modifythe sampling rate based on the movement count 1040 the user is on whileperforming a set of exercises. For example, if a user is expected toperform a set of 5 curls, the application 19 may increase the samplingrate as the user approaches count 4 and count 5 1040 of the set ofcurls. This allows, the application 19 to better capture changes in thevalue of processed signal 1 1010 and 2 1020, thereby allowing theapplication 19 to better determine whether the user demonstrated strainwhile performing the last few counts of a set of exercises. Theapplication 19 may also modify the sampling rate in other situationswhere a more refined or accurate measure of the motion being performedby the user would help the application 19 better understand the motionsbeing performed by the user.

FIG. 11 illustrates the application 19 dynamically adjusting the workoutsuggested to the user, according to one embodiment. The application 19dynamically modifies the workout suggested to the user based on themotion data 25, the number of repetitions performed by the user and/orinstances of strain identified by the user. For example, the user mayaim to reach a workout goal 1120 comprising 15 squats in the first set,13 squats in the second set and 13 squats in the third set. The oldrepetition count 1110 monitored by the application 19 is 14 squats forthe first set, 12 squats for the second set and 8 squats for the thirdset. The new repetition counts 1130 performed by the user may be 15squats for the first set, 12 squats in the second set and 13 squats inthe third set. The application 19 identifies that the user has improvedthe number of repetitions in the third set from 8 squats to 13 squats,and dynamically adjusts 1140 the number of repetitions in the workoutgoal in the third set from 13 to 15 (not shown in FIG. 11). Theapplication 19 thereby monitors and facilitates the physical improvementof the user, without the user having to provide user input 11 to theapplication 19 via the smart device 18 or the motion tracking device 24.Once the user has finished a set of exercises, the application 19 maymonitor and inform the user of rest time between exercises by displayingto the user a rest timer 1150, for example.

In another embodiment, the application 19 dynamically adjusts 1140 theworkout suggested to the user based on the application 19 identifyingthat the user has demonstrated strain while performing an exercise. Forexample, the application 19 may identify a user demonstrating strain inthe 5^(th) movement count of a set of 10 curls, and may then dynamicallyadjust 1140 the workout suggested by the application 19, by reducing theweight suggested for the next set of curls. In another example, theapplication 19 may reduce the number of repetitions performed per set ofcurls if the application 19 identifies the user demonstrating straintowards the end of each set of curls.

In one embodiment, the application 19 determines how to adjust 1140 theworkout of the user based on information received from the trainerprescribing the workout or the combination of trainers prescribing theworkout. For example, referring again to FIG. 6, the application 19 mayidentify, based on the motion data 25, that the user is unable tocomfortably perform the workout suggested by trainer 1 615. It ispossible that the application 19 identifies that the user isconsistently underperforming when it comes to completing the number ofrepetitions suggested by trainer 1's 615 prescribed workout. Theapplication 19, would then adjust 1140 the workout suggested by theapplication 19 based on a training information representing thetechniques and viewpoints of a specific trainer. In one example, trainer1 615 may suggest that the user perform fewer repetitions but maintainthe weights currently prescribed by the workout, as trainer 1specializes in suggesting workouts that improve the strength of theuser. Trainer 2 however, given the same motion data 25 and information,may reduce the weights but increase the repetitions as trainer 2specializes in suggesting workouts that improve the tone of the user.Thus if the user is performing a workout suggested by trainer 1 615, theapplication 19 may automatically adjust the workout based on thetraining information representing trainer 1.

The application 19 can also dynamically adjust the workout based onadditional data such as the amount of sleep the user had the previousnight. If the user did not get sufficient sleep, which may be relativeto the typical sleep pattern of the user, then the application 19 maydynamically reduce the intensity of the workout. Determining the amountand/or quality of sleep can be based on receiving an input from the useror can be based on measurements by sensors, e.g., sensors 6, 7, 8 of themotion tracking device 24.

In one embodiment, the application 19 dynamically adjusts 1140 theworkout of the user over multiple days based on the motion data 25. Forexample the application 19 may determine that the user has improvedsubstantially compared to the workout goals set for exercises to beperformed on Monday, and may alter the exercises or the repetitions andweights associated with exercises for subsequent days of the workoutsuch as those exercises performed on Tuesday.

In one embodiment, the motion tracking system 100 dynamically turnssensors on or off. The application 19 may turn sensors on or off basedon the exercise being performed by the user. For example, theapplication 19 may be able to identify and monitor a curl using motiondata 25 from the accelerometer 6. In this instance when the user isperforming a curl the application 19 may turn off the gyroscope 7 andthe magnetometer 8 so as to minimize the power consumed while runningthe application 19.

In another embodiment, the application 19 may turn on or off sensorsbased on the count of the exercise the user is currently performing. Forexample, the application 19 is aware that determining whether user isdemonstrating strain occurs generally towards the end of a set. Theapplication 19 may perform the identifying and monitoring of the curlmotion using motion data 25 from the accelerometer 6, and may performidentifying strain using a combination of motion data 25 from theaccelerometer 6 as well as the gyroscope 7. The application 19 may thenturn off the gyroscope 7 for counts 1 through 7 of a set of 10 curls,and may then turn on the gyroscope 7 as the user begins to perform count8 of the set of curls. Similarly the application 19 may turn on or offsensors based on the portion of a repetitive motion the user isperforming. For example, the application 19 may turn off the gyroscope 7as the user raises a dumbbell while performing a curl, and then theapplication 19 may turn on the gyroscope 7 as the user begins to lowerthe dumbbell while performing a curl.

In one embodiment, the application 19 may prompt the user to calibratethe motion tracking system 100. The application 19 may suggest that theuser perform a set of calibration motions as the application 19 monitorsthe movements of the user. As the user performs the calibration motionsthe application 19 may identify user specific motions, and hence modifythe movement templates 27 representing good form or improper form in themovement template database 22 based on the recorded calibrationmovements, e.g., the calibration can assist in identifying the signalamplitude of various movements. In one embodiment, the user maycalibrate the motion tracking system 100 as a user performs a workout.For example, the system may calibrate while the user performs a curl assuggested by a workout selected by the user.

In one embodiment, the motion tracking system 100 may be used as anartificial coaching aide. Coaches may use the motion tracking system 100to monitor the workouts performed by athletes over a period of time. Themotion tracking system 100 may also be used to monitor the form of anathlete while performing a sporting activity, such as a golf swing or athrowing a baseball. For example, a golf coach may use the motiontracking system 100 as an artificial teaching aide while providing agolf lesson or for reviewing a previous golf lesson. If the user or astudent drops the club face in his back swing, the motion trackingsystem 100 may provide feedback to the user, notifying the user of thetype of bad form displayed and ways the user can improve. Hence, thecoach can actively provide feedback to the athlete or studentirrespective if the coach and student are present at the same location.The coach can also monitor the student's progress and responsiveness todifferent coaching styles, and adapt his/her coaching style to bettersuit the needs of the student.

In one embodiment, the coach can receive fitness data representing thefitness of the user or athlete based on the motions performed by theuser, as monitored by the motion tracking system 100. For example, theapplication 19, may notify the user and coach of fatigue during a game,if the application 19 notices that the user is consistentlyunderperforming physically during a game or training session. Theapplication 19, may notice that one or more signals representing amotion no longer contains peaks or valleys with amplitudes similar tothat present in historical data representing a good or proper motion,hence indicating that the user may be demonstrating fatigue.Understanding fatigue may help the coach prevent potential injuries toan athlete during a game or training session.

In one embodiment, the coach and user or athlete may use the motiontracking system 100 to monitor tasks performed by a user. For example,the motion tracking system 100 may be used to monitor the number ofshots attempted by a basketball player during a training session or agame. In another example, the coach may monitor and generate a set ofquantitative performance metrics related to an athlete or user. Thecoach may determine the power generated by an athlete based on thenumber of repetitions of an exercise performed by the athlete over aperiod of time. The coach may also be able to monitor other metrics,like average distance run, calories burned or shots attempted by anathlete during a training session or a game. This reduces the amount ofinfrastructure used while monitoring the progress of an athlete.

In one embodiment, the motion tracking system 100 and the application 19may used as an artificial medical aide for a physical therapists. Thephysical therapist may monitor the motion and movements of one or moreof his/her patients to better understand the affects of the therapybeing offered. In one embodiment, the physical therapist may monitoreach user's response to different kinds of therapy to better understandthe affect of a kind of therapy over a variety of users. For example, aphysical therapist may be treating a number of users who all suffer fromthe same ailment. The physical therapist may try different treatments ondifferent users. The physical therapist may then use the motion trackingsystem 100 to monitor the progress and improvement of each of the usersduring the course of the therapy to better understand which therapieswere more effective and how user's responded to each therapy.

In one embodiment, the physical therapist is able to better understandwhat therapy to prescribe to a user based on the motion data provided bythe application 19 and the motion tracking system 100. For example, thetherapist is able to monitor the number of steps walked by a user in aperiod of time, the amount and kind of motions performed by the userover a period of time and the number of repetitions and form of themotions performed by a user over a period of time. The physicaltherapist may also be privy to motion data 25 indicating that the useris demonstrating strain while performing various motions during a periodof time. Hence, the physical therapist may be able to provide the userwith better therapy options after getting a holistic understanding ofthe user's fitness.

In one embodiment, the motion tracking system 100 may be programmed tonotify the user when the user is over exerting himself/herself asdetermined by the physical therapist. For example, the application 19may identify that a user with a pulled hamstring is generating motiondata 25 while performing leg extensions, where the average amplitude ofpeaks in one or more recorded signals is greater than an amountsuggested by the physical therapist. The application 19 would thenprovide the user with feedback notifying the user that he/she is overexerting himself/herself and is likely to aggravate the injury.

In one embodiment, the application 19 may detect injury in one or moremotions performed by the user and provide the user with feedbackassociated with alleviating an injury. For example, the application 19may identify that the user is walking with a motion similar to that of asprained ankle. The application 19 may then provide the user withfeedback advising the user of how to alleviate the injury, and providethe user with contextual information such as where the nearest hospitalis, or whether the user would like to call an emergency contact forhelp.

In one embodiment the application 19 may use other contextual data 26 toprovide a better fitness experience to the user. For example theapplication 19 may use geographical information from a GPS service onthe smart device 18 to determine the location of the user, and providethe user with fitness related supporting contextual information. In oneinstance, the application 19 via the GPS service may identify that theuser is at a restaurant. The application 19 may then provide the userwith options available on the menu of the restaurant that best suit thenutritional needs of the user based on the overall fitness of the useror on the current workout being performed by the user. For example, auser subscribed to a strength training workout may receive entréesuggestions with high protein content.

In another embodiment, the application 19 based on data received fromthe GPS service on the smart device 18, may identify the gym the user isworking-out in and modify the workout suggested to the user based onequipment available at the gym. For example, an application 19 mayidentify that the user is in a gym without free weights. The application19 may then modify the workout suggested to the user for that session,by replacing the free weight exercise with exercises the user mayperform using the equipment available in the gym and still substantiallymeet the goals of the workout. In one embodiment the application 19 maymodify a workout suggested to a user, based on altitude informationreceived from the GPS service on the smart device 18 or from analtimeter providing altitude information to the application 19. Forexample, based on the altitude of a user's current location theapplication 19 may suggest that the user run a shorter distance if thealtitude identified by the application is substantially greater than theaverage altitude the user is acclimated to.

In one embodiment, the application 19 may dynamically reschedule aworkout based on a calendar information received from a calendar servicepresent on the smart device 18. The application 19, may actively movetimes on a user's calendar for when workouts are scheduled based onother conflicting appointments that may arise. For example, the user maynot be able make a workout session scheduled for 8 AM due to a businessmeeting the user would like to attend. The application 19 may reschedulethe workout for 8 PM, the next free slot determined by the application19 in the user's calendar. In one embodiment, the application 19 mayschedule a workout for another day, if a user is unable to perform thecurrent workout due to other appointments.

In one embodiment, multiple devices providing additional sensorinformation, e.g., additional fitness information, may be integratedwith the motion tracking system 100. The application 19 may receivefitness information from other devices monitoring different fitnessaspects of the user. For example, the application 19 may receiveinformation from a heart rate monitor 38 representing the user's heartrate over a period of time and/or in real-time. The application 19 mayalso receive fitness information from equipment using electromyographyto monitor the movements associated with a user's muscles. Integratinginformation from multiple devices can assist the application 19 byproviding addition information that can be used to better identify theoverall fitness of the user. The application 19 may also modify workoutssuggested to the user based on information received from the integrateddevices. For example, the application 19 may reduce the number ofrepetitions or intensity of a workout suggested to the user if theapplication 19 identifies that the user's heart rate is unusually higherthan normal.

In one embodiment, the application 19 may receive fitness informationfrom a posture monitoring device, or a combination of devices thatmonitor the user's posture. The application 19 may then provide feedbackto the user notifying the user of bad posture or bad form whileperforming a motion or activity. For example the application 19 mayreceive information indicating that the user is maintaining bad posturewhile sitting down at work. The application 19 may notify the user ofthe bad posture and demonstrate visually to the user a representation ofgood posture.

In one embodiment, the motion tracking system 100 allows for one or moreusers to track the workout and progress of other users. For example, auser may choose to connect their workout with one or more differentusers, by making their user profile 29 accessible to other users. In oneinstance, this allows for users to have asynchronous workout buddies, asboth users may perform the same workout at different locations and atdifferent times and still be able to keep track of each other's progressand improvement. In another example, competitors of a competition or agame may be monitored using the motion tracking system 100. For example,the competitors of marathon may all share their user profile 29 with thecompetition hosts. The competition hosts, may then accurately monitorthe time taken for each user to finish the marathon, irrespective of thetime of day or the starting group each user starts in.

In one embodiment, the motion tracking system 100 may be used for injuryprevention while a user is performing a specific task at work or ingeneral. For example, an owner of a freight company may monitor thenumber of times each of his workers lifts a set of boxes and moves theboxes a certain distance. The owner may set a limit on a maximum numberof lifts a worker should perform so as to prevent an injury. Theapplication 19 may then notify the worker when the worker has reachedthe maximum number of repetitive lifts and may ask the worker to performno more lifts.

In another embodiment, the motion tracking system 100 may be used tomonitor movements performed by workers performing routines while on thejob. This may help managers monitor employees demonstrating strainduring a routine and determine the particular patterns or movements inthe routine that cause the employees to demonstrate strain.

In one embodiment, the motion tracking system may be used to identifyrecoil as a user uses a firearm. This may help the user or a supervisingbody monitor the number of rounds fired by the user over a period oftime. In one example, the supervising body may monitor the performanceof the user during a simulation, based on the user's movements duringthe simulation and the number of rounds fired by the user. In anotherembodiment, a firing range may accurately monitor the number of roundsfired by a user, and charge the user an appropriate price.

In one embodiment, the application 19 may identify a gesture or a set ofgestures performed by a user and take an appropriate action related tothe gesture performed by the user. For example, the application 19 basedon the motion data 25 may identify that while at a networking event auser wearing a motion tracking device 24 shakes the hand of a seconduser also wearing a motion tracking device 24. Based on the patternsassociated with a hand shake gesture and the user profile 29 associatedwith the second gesture, the application 19 may perform the action ofsending a connection invite to one or more profiles of the second useron one or more social networking websites. Hence, a physical motionperformed by the user, such as a handshake, may be used as a key toenable a device or program to perform an action, such as pairing twodevices or sending a social networking invitation.

In another embodiment, the application 19 may identify gestures relatedto the user interacting with a set of services available on the smartdevice 18. For example, the user may receive an email on the smartdevice 18. The user may open the email by performing a circular rotationof the wrist (the gesture) resulting in the application 19 identifyingthe gesture, and taking action by opening the email on the smart device18.

In one embodiment, the application 19 may be used to monitor routinesperformed by a user, where the routine is not a fitness routine. Forexample, the application 19 may be used to monitor the steps performedby a surgeon as the surgeon performs a complex surgery. The application19 may monitor the actions or movements performed by the surgeon, andwarn the surgeon if the surgeon skipped a step in the routine. Similarlythe motion tracking system 100 may be used to monitor industrialroutines performed by industrial workers. For example, a routineperformed by a worker may involve the worker performing a motion A, thena motion B and then a motion C, after which the worker repeats thecycle. The application 19 may warn or notify the user if the user misseda particular motion in the routine. Furthermore, the application 19 maygenerate quantitative metrics representing efficiency of the worker asthe worker performs a routine or the number of times the workerperformed the routine. Hence, the motion tracking system 100 is notlimited to tracking the fitness routine of a user.

In one embodiment, the motion tracking system 100 may count therepetitive motions performed by a user as a user performs an activity,the activity not being limited to a fitness activity. For example, theapplication may monitor the number of times a surgeon picks up one ormore sponges and places the sponges around an open wound or area of apatient's body while the surgeon is performing a surgery on the patient.The motion tracking system 100, based on the count of the number ofsponges placed by the surgeon at the beginning of the surgery, maynotify the surgeon if the surgeon forgets to remove the same number ofsponges from the patient's body towards the end of the surgery. Hence,the motion tracking system is not limited to identifying and monitoringrepetitive motions related to a fitness activity or routine.

Reference in the specification to “one embodiment” or to “an embodiment”means that a particular feature, structure, or characteristic describedin connection with the embodiments is included in at least oneembodiment. The appearances of the phrase “in one embodiment” or “anembodiment” in various places in the specification are not necessarilyall referring to the same embodiment.

Some portions of the detailed description are presented in terms ofalgorithms and symbolic representations of operations on data bitswithin a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of steps (instructions)leading to a desired result. The steps are those requiring physicalmanipulations of physical quantities. Usually, though not necessarily,these quantities take the form of electrical, magnetic or opticalsignals capable of being stored, transferred, combined, compared andotherwise manipulated. It is convenient at times, principally forreasons of common usage, to refer to these signals as bits, values,elements, symbols, characters, terms, numbers, or the like. Furthermore,it is also convenient at times, to refer to certain arrangements ofsteps requiring physical manipulations or transformation of physicalquantities or representations of physical quantities as modules or codedevices, without loss of generality.

However, all of these and similar terms are to be associated with theappropriate physical quantities and are merely convenient labels appliedto these quantities. Unless specifically stated otherwise as apparentfrom the following discussion, it is appreciated that throughout thedescription, discussions utilizing terms such as “processing” or“computing” or “calculating” or “determining” or “displaying” or“determining” or the like, refer to the action and processes of acomputer system, or similar electronic computing device (such as aspecific computing machine), that manipulates and transforms datarepresented as physical (electronic) quantities within the computersystem memories or registers or other such information storage,transmission or display devices.

Certain aspects of the embodiments include process steps andinstructions described herein in the form of an algorithm. It should benoted that the process steps and instructions of the embodiments can beembodied in software, firmware or hardware, and when embodied insoftware, could be downloaded to reside on and be operated fromdifferent platforms used by a variety of operating systems. Theembodiments can also be in a computer program product which can beexecuted on a computing system.

The embodiments also relate to an apparatus for performing theoperations herein. This apparatus may be specially constructed for thepurposes, e.g., a specific computer, or it may comprise ageneral-purpose computer selectively activated or reconfigured by acomputer program stored in the computer. Such a computer program may bestored in a computer readable storage medium, such as, but is notlimited to, any type of disk including floppy disks, optical disks,CD-ROMs, magnetic-optical disks, read-only memories (ROMs), randomaccess memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards,application specific integrated circuits (ASICs), or any type of mediasuitable for storing electronic instructions, and each coupled to acomputer system bus. Memory can include any of the above and/or otherdevices that can store information/data/programs and can be transient ornon-transient medium, where a non-transient or non-transitory medium caninclude memory/storage that stores information for more than a minimalduration. Furthermore, the computers referred to in the specificationmay include a single processor or may be architectures employingmultiple processor designs for increased computing capability.

The algorithms and displays presented herein are not inherently relatedto any particular computer or other apparatus. Various general-purposesystems may also be used with programs in accordance with the teachingsherein, or it may prove convenient to construct more specializedapparatus to perform the method steps. The structure for a variety ofthese systems will appear from the description herein. In addition, theembodiments are not described with reference to any particularprogramming language. It will be appreciated that a variety ofprogramming languages may be used to implement the teachings of theembodiments as described herein, and any references herein to specificlanguages are provided for disclosure of enablement and best mode.

In addition, the language used in the specification has been principallyselected for readability and instructional purposes, and may not havebeen selected to delineate or circumscribe the inventive subject matter.Accordingly, the disclosure of the embodiments is intended to beillustrative, but not limiting, of the scope of the embodiments, whichis set forth in the claims.

While particular embodiments and applications have been illustrated anddescribed herein, it is to be understood that the embodiments are notlimited to the precise construction and components disclosed herein andthat various modifications, changes, and variations may be made in thearrangement, operation, and details of the methods and apparatuses ofthe embodiments without departing from the spirit and scope of theembodiments as defined in the appended claims.

What is claimed is:
 1. A computer based method for interpretingrepetitive movements performed by a user of a motion tracking device aspart of updating a physical therapy treatment, the method comprising:receiving motion data from the motion tracking device, the motion datarepresenting motions performed by the user of the motion tracking deviceassociated with a repetitive movement associated with the physicaltherapy treatment; determining, by the computer, based on the motiondata and the repetitive movement, a form with which the user performsthe repetitive movement having the steps of: identifying, by thecomputer, repetitive features in one or more signals of the motion data;comparing, by the computer, the repetitive features with correctmovement data associated with the repetitive movement; and determining,by the computer, based on the comparison, the form with which the useris performing the repetitive movement including the steps of:identifying, by the computer, repetitive features in one or more signalsof the motion data; comparing, by the computer, the repetitive featureswith correct movement data associated with the repetitive movement; anddetermining, by the computer, based on the comparison, the form withwhich the user is performing the repetitive movement; generating, by thecomputer, based on the determined form with which the user performs therepetitive motion, feedback, the feedback representing the form withwhich the user is performing the repetitive movement; receiving anupdated physical therapy treatment based upon the determined form. 2.The computer based method of claim 1, wherein determining, based on therepetitive features, the type of improper form being performed by theuser comprises: comparing the repetitive features with one or moreimproper movement data associated with the repetitive movement; anddetermining, based on the comparison, the type of improper form beingperformed by the user.
 3. The computer based method of claim 1, whereinthe updated physical therapy treatment is modified to provide the userwith better therapy options in response to the motion data.
 4. Anon-transitory computer-readable storage medium storing a computerprogram having instructions encoded thereon that, when executed by aprocessor, cause the processor to: receive motion data from the motiontracking device, the motion data representing motions performed by theuser of the motion tracking device associated with a repetitive movementassociated with the physical therapy treatment; determine, based on themotion data and the repetitive movement, a form with which the userperforms the repetitive movement comprising: identify repetitivefeatures in one or more signals of the motion data; compare therepetitive features with a correct movement data associated with therepetitive movement; and determine, based on the comparison, the formwith which the user is performing the repetitive movement including:identify repetitive features in one or more signals of the motion data;compare the repetitive features with a correct movement data associatedwith the repetitive movement; and determine, based on the comparison,the form with which the user is performing the repetitive movement;generate, based on the determined form with which the user performs therepetitive motion, feedback, the feedback notifying the user of the formwith which the user is performing the repetitive movement; and receivean updated physical therapy treatment based upon the determined form. 5.The non-transitory computer-readable storage medium of claim 4, whereindetermine, based on the repetitive features, the type of improper formbeing performed by the user comprises: compare the repetitive featureswith one or more improper movement data associated with the repetitivemovement; and determine, based on the comparison, the type of improperform being performed by the user.
 6. The non-transitorycomputer-readable storage medium of claim 4, wherein the updatedphysical therapy treatment is modified to provide the user with bettertherapy options in response to the motion data.